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Abstract. The 9-Neumann operator (the inverse of the complex Laplacian) 
is shown to be noncompact on certain domains in complex Euclidean space. 
These domains are either higher-dimensional analogs of the Hartogs triangle, 
or have such a generalized Hartogs triangle imbedded appropriately in them. 



1. Introduction 

Let ni, n 2 be positive integers, and let n = ni+n^- For a point z = [z\, . . . , z n ) G 
C™, we write ' z for the point (zi, . . . , z ni ) in C™ 1 , and also write z' for the point 
(z„ 1+ i, . . . , z n ) in C™ 2 , and denote z — ('z, z'). Let a > 0, and consider the bounded 
pseudoconvex domain H in C™ given by 

={('*,/) eC»: ||'z|| (1) <(||z'|| (2) ) Q <l}, (1) 

where ||'||m, || -|| ( 2 ) are arbitrary norms on the complex vector spaces C™ 1 , C" 2 
respectively. 

We may refer to the domain H of ([1]) as a Hartogs Triangle, a term usually 
applied to the case rii = rz.2 = a = 1. This domain H belongs to the class of 
domains for which non-compactness of the 9-Neumann problem is established in 
this note. (See [H] for the relevant definitions, as well as a comprehensive discussion 
of compactness in the 9-Neumann problem. Other important texts dealing with 
the (9-Neumann problem include [HH].) To define precisely the class of domains we 
will be considering, note that the boundary bH of H contains the piece 

S={(' Z) z')eC n : IN (1) <i,||/|| (3) =i} 

= B 1 xbB 2 , (2) 

where Bj C C™ 1 and B 2 C C" 2 are the unit balls in the norms \\'\\(u and II " II (2) 
respectively. In this note, we prove non-compactness of the 9-Neumann operator on 
a bounded pseudoconvex domain Q in C™, which has roughly speaking the following 
property: it is possible to embed the Hartogs triangle H of (fl]) into f2 in such a way 
that the subset S of the boundary of H given in ([2]) is mapped into the boundary 
of Q. For example, it suffices to assume that there is a biholomorphic map from a 
neighborhood of H into C", which maps H into 17 and S into bO. 

More precisely, it is sufficient to assume the following hypotheses on fi: (i) there 
exists a biholomorphic map 

F :H -> F(H) C Q, (3) 
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which extends to a C°°-diffeomorphism of a neighborhood of H onto a neighborhood 
of F{H) in C". (ii) Further, F itself extends biholomorphically to S in such a way 
that 

F(S) C bO. (4) 
For such f2, we will prove the following: 

Theorem. For 1 < q < n% the d-Neumann operator of O acting on L\ _(fi) is 
non-compact 

Acknowledgments: The author thanks Professors Mei-Chi Shaw and Emil Straube 
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2. Some remarks 

We note that for each £ € bB2, the subset F(S) C 6i7 of @ contains the rii- 
dimensional analytic variety F(M>i x {£}), and this means absence of the classical 
compactness-entailing property (P q ), for 1 < q < n\. This strongly suggests the 
noncompactness of the 9-Neumann operator in degrees (0, q) for 1 < q < n\. On the 
other hand, (P q ) is not known to be necessary for compactness, so this observation 
by itself does not show that the 9-Neumann operator is noncompact. When n-x = 1, 
so that the boundary bfi contains n — 1 = ni dimensional complex manifolds, the 
noncompactness can be deduced from a result of Catlin, according to which the 
presence of such [n — l)-dimensional manifolds on the boundary of a weakly pseu- 
doconvex domain in C™ implies non-compactness of the d-Neumann operator on 
(0, n— 1) forms (see [5], where it is assumed that n = 2, but the boundary is allowed 
to be Lipschitz.) In the analogs of this result for higher-codimensional manifolds in 
the boundary (see [TTJ Theorem 4.21]) one needs to assume that the boundary is 
strictly pseudoconvex in the directions transverse to the complex manifold. This is 
not true in general for the generalized Hartogs triangle of ([I]), since e.g., we can take 
1 1 11(2) = max i<i<n2 \ w 2\- In general, establishing compactness is a tricky business 
(see [TTJ Chapter 4].) Our interest in the rather special domains fi stems from the 
fact that we are able to demonstrate noncompactness by an explicit and elemen- 
tary counterexample in the spirit of [SJ [TO] and [5J Proposition 6.3] by exploiting 
the symmetry of Q inherited from the rotational symmetry of H. 

We also note here that the result can also be stated in the situation when is a 
relatively compact Stein domain in an n complex-dimensional Hermitian manifold. 
Under the hypothesis of existence of a map F with the same properties as above, 
we can prove the noncompactness of the <9-Neumann operator on Lp q (il), where 
< p < n and 1 < q < n\. The changes required in the proof are purely formal, 
and for clarity of exposition we stick with the ambient manifold C™. 

Let iV be a positive integer, for 1 < j < N, let rij be a positive integer, and 
let n = 5Zj=i n j- Suppose that we fix a norm IHly) 011 the vector space C™ 3 , and 
denote a point z € C n by (z^\ . . . , z^ N '), where z^' € C lj . In principle, it should 
be possible to modify the proof of the theorem, to prove the noncompactness of the 
9-Neumann operator on a domain of the form 



(i) 



< 



,(2) 



(2) 



< 



,(3) 



(3) 



< 



< 



Z (N) 



(N) 



< 1 
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or more generally, on a domain in which H is appropriately embedded. The expo- 
sition in the general case will involve complicated notation arising from the more 
intricate geometry, and for clarity, we write the proof in a simple situation. 



3. Preliminaries 

Pulling back the canonical metric of C™ via the map F _1 (with F as in (J3J), 
followed by a partition of unity argument, there is a smooth Hcrmitian metric h 
on St, such that the map F is an isometry on H . We use the metric h to define 
the pointwise inner product on spaces of forms, and the volume form on SI in 
the standard way, and let L 2 , q (Sl) be the L 2 -space of (p,q)-iorms defined by this 
pointwise inner product and volume form. Thanks to [1] Theorem 1], we know 
that whether or not the 9-Neumann operator is compact on the bounded domain 
SI is independent of the hermitian metric on Si, as long as the metric is smooth 
on SI. Consequently, it will suffice to show that the d-Neumann operator defined 
with respect to the metric h acting on L,Q q (Sl), (denoted by N g ) is noncompact, 
for 1 < q < n\. Since N g +i is compact if N g is compact for 1 < q < n — 1, 
(see [TTJ Proposition 4.5]) it follows that it is sufficient to show the operator N Ml : 
-^o m (^) ~ ^ ni (Si) is non-compact. 

Let d denote the Hilbert-space adjoint of d on L\ n (SI) with respect to the 
metric h, and denote by fj the space 

Dom(d) n Dom(9*) C L 2 Q ni {SI), (5) 

which is a Hilbcrt space with norm 



,2 



/IU = ( WWti mi+l(n ,+ of .„ j ■ (6) 



2 



^0, rei -l( fi ) 



To prove non-compactness of N ni , it is sufficient to show that the inclusion map 
of $) into Lq n (Si) is noncompact ([11, Proposition 4.2].) Before we proceed to 
show this we compute certain quantities which will be useful in the proof. We will 
denote the (2n2 — l)-dimensional Hausdorff measure induced on a hypersurface in 
R 2 " 2 — C™ 2 by da. For v > 0, we consider the function 

7(1/)=/ \U 2v da(t), (7) 

JbK 2 

where t = (ti,... ,t ri2 ) £ C" 2 , and the integral extends over the boundary bB 2 = 
{||t|| (2) = 1} of the unit ball B 2 of the norm ||-|| (2) on C" 2 . We will denote by dV 
the Lebesgue measure on Euclidean spaces of arbitrary dimensions (the dimension 
being known from the context.) We first prove a couple of lemmas: 

Lemma 1. For /3 > 0, we have [ \w n2 \ 21 ' lliollw, dV(w) — — — 

Jk 2 (> 2(v + p + n 2 ) 

Proof. Using the co-area formula, we can rewrite the integral on the left as 

(/ \w n2 \ 2v r 2 ^da(w) ) dr, 

/r=0 \J{\\w\\ (2) =r} J 

where the inner integral extends over the level set {||u>|l(2) = r } c ^-™ 2 ano - * s 
with respect to the surface measure a induced on this set by the ambient Lebesgue 
measure. To evaluate the inner integral, we make the substitution w = rt, where 



4 



DEBRAJ CHAKRABARTI 



11*11(2) = 1' i- e -' * nes m t ne boundary bB2 of the unit ball of the ||-||( 2 ) norm. Taking 
advantage of the fact that the level sets of the norm are related by dilations, we 
can compute this integral as 



r 2v '\t n2 \ 2h W" 2 - 1 ^)) dr 

1 r 2v+2 P+ 2n 2 -l dr \( f \ tn2 \^ da{ t) 
J \Ab 2 



7(l/). 



2(v + [3 + n 2 ) 

□ 

We will need the following fact regarding removable singularities (cf. [7]): 

Lemma 2. Let N > 3, and Zei P be a first order linear partial differential operator 
on a domain U C R N . Let q £ U , and suppose that u,v £ L 2 {U) are such that 
on U \ {q} we have in the sense of distributions Pu — v. Then, as distributions, 
Pu = v on U . 

Proof. For e > 0, let Xe £ Cq°(R n ) be a smooth cutoff such that < Xe < 1, Xe is 
equal to 1 near q, Xe vanishes outside the ball B(q, e), and we have |Vxe| = 0(e^ 1 ). 
Let 4> G Cq°(U)- Writing cf) = (1 — Xe)4> + Xt4>, an d denoting the adjoint of P by P* 
we have 

\{Pu-V^)\ = \{Pu-V, X e<f>)\ 

< \{u,P*{ X e<l>))\ + \{v,Xe<t>)\ 

< C (e- 1 + 1) ^oKB(^ejj 
-> as e -> + , 

which proves the lemma. □ 

We note here that although Lemma[5]is stated for operators acting on functions, 
the result, as well as the proof (after formal changes) continue to hold for differential 
operators like d that act on sections of vector bundles. 

4. Counterexample to compactness of N ni 

We now construct a sequence {«„} bounded in .f) which has no convergent subse- 
quence when viewed as a sequence in Lq (f2). Let x be a real- valued non- vanishing 
smooth function on the interval [0, 1] which vanishes in a neighborhood of 1. Denote 
by (zi, Z2, ■ ■ ■ , z n ) — ('z, z') £ C™ 1 x C™ 2 the coordinates on F{H) c given by the 
map F^ 1 : F(H) — > C". For each positive integer u we consider the (0,ni)-form 

(\\' z \\ \ ni 

wy^wij""-^ mF{H) - (si 

k 0, elsewhere. 

(Here a is as in (jTJ) , and 7 is as in ([7}) Then has support in F(iJ) , is bounded on 
Q, and is smooth everywhere on f2 except at F(0). In particular, each u v £ L\ (fl). 
We claim that the sequence is bounded in io but has no convergent subsequence 
as a sequence in L\ n (Q). 
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Before we begin the proof of the claim, we collect a couple of simple compu- 
tations. Note that a norm ||-|| on the vector space is C fe is Lipschitz, and hence 
differentiable almost everywhere, and it is easy to see that its gradient is bounded, 
i.e. there is a constant K\ > 0, depending only on the norm ||-|| such that 

|V {\\z\\)\ < K t , for almost every z £ C fc , 

where |-| denotes the Euclidean norm in C k . Now, a computation shows that 



V X 



'\Y4a 



(i) 



X 



(2), 



where the gradient is taken in 
for any z £ H, we have 

V ( X 

We now estimate \\du u 




Therefore there is a constant K2 > such that 




< 



K? 



(9) 



1(2) 



L a , nv Since u u £ L% by Lemma [2] the sin- 

gularity at F(0) of the form u v can be ignored while computing du v , provided 
du v £ Lq n Since z^ is holomorphic, we have on F(H): 



du u = 



7W 



<-d X 




From now on, C denotes a constant independent of is, which may be different at 
different occurrences of the symbol. Using ©, we have, 

V ix ' 



7(1/) 




(10) 



L 2 (F(H)) 



< c 



7W 



1(2) 



(11) 



LHH) 



To compute integrals over the Hartogs Triangle, we will use the new coordinates 
(v,w) induced by the homeomorphism $ from Bi x (B 2 \ {0}) C C™ to H C C" 
given by 

$(v,w) = (\\w\\« 2) v,w), (12) 

which is differentiable a.e., and which has Jacobian determinant ||u>||^ ni a - e - f^ e " 
calling that dV denotes the Lebesgue measure on Euclidean space, we have 
2 . .0.. 



Ik'll 



(2) 



'-^dV(z) 

H |M|(2) 



lX (B 2 \{0}) ||HI(2) 



\ w \\(2) ni dV{v,w) 



Vol(Bi) 
Vol(Bi) 



w 



i2ce(«i— 1) 
1(2) 



dV{w) 



7(1/) 



2{v + a{n x - l) + n 2 )' 



(13) 
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where we have used the lemma proved in Section [3] Combining (fTTj) and (p~3|) gives 
the estimate 



\du v \ 



l(") 



< C 



v + a(ni - 1) + n<x ' 

hence combined with the fact that u u £ L\ (f2) each u v is in the domain Dom(<9) 
of the Hilbert space operator <9, and there is a C independent of v such that 



\du v , 2 ro s < C. 

1 "^O.nj + l^'J 



(14) 



Let i? denote the formal adjoint of d on h\ n (O) with respect to the metric /i 
defined above. Then on F(H), in the coordinates (zj., . . . , z n ), the formal expression 
for i? coincides with the usual one in Euclidean space. If 



U v {z) = 



l(i) 



(15) 



1(2), 



so that on F(H) we have = \J v dz\ A • • • A cfz^7, we have on F(H) 



3 = 1 



A dzj A 



c9 



7(") " £^ 



j=i 




dzi A 



A A 



where the hat on dzj denotes the omission of this factor from the wedge product. 
Therefore, using ^ we compute 



\\#u»\ 



H -i(n) 



< C 




L 2 (F(H)) 



which is the same quantity as in (1101) . Therefore, the same arguments as used 
before for du v show that "du v £ Lq and indeed is uniformly bounded in 

Lq „ i(J2) independently of v. 

Further, the form u v vanishes everywhere on the boundary Wl except in the patch 
bO n F(H), which consists of the disjoint union of the sets F(S) and {F(0)} n bCl, 
with F as in ©. (Note that {F(0)} n bfl is empty if F(0) G Q.) Near F(5) 
the boundary is represented in the local coordinates ('z, z') g C" 1+ " 2 as {('z, z') g 
C™ : 1 1 z' 1 1 ( 2 ) = !}■ From the formula ([8]) we see that the complex- normal component 
of u v vanishes on bO D F(S). For e > 0, let Xe be a cutoff of the type used in the 
proof of LemmaH < Xe < 1> Xe = 1 n ear ^(0), Xe vanishes outside B(F(0),e) 
and |VXe| — 0(e^ 1 ). Note from the definition that u v has bounded coefficients. 
Writing ip e = 1 — % e we have 

d{tp t u u ) = U v {z) ■ - —-dzj A • • • A dzj A • • • A dzJ7 + 4> t "t}u v , 



3 = 1 



where {/„ is as in (fT5|) on F(H) and extended as zero elsewhere. The second term 
approaches $it„ in the L 2 -topology as e — s- + and the L 2 -norm of the first term is 
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bounded by 

C \\U»\\ L «, m e-VVol(B(F(0),e)), 

which goes to as e — > + . Note that tp e u v is in Dom(9 ), since it is a smooth 

(0,ni)-form on CI whose complex-normal component vanishes along F(S) C bfi, 

and %l) e u u itself vanishes elsewhere on the boundary. Further, i\) t u v — ¥ u v in the 

graph norm of It follows that u v G Dom(<9 ), and from the computation above, 

— * 

d Uv < C, where C does not depend on v. Combining this with (|14|). 

i O,,ii-l( f2 ) 

it follows that each u v lies in the space i!) of (|5]), and we have 

IKIU<c- 

We now compute ||u ! /|| i 2 We again use the change of coordinates given by 

H: 

Kll£* (fi) -~ 




~T\ I X Z (\\v\\(i))\wn 2 \ 2 " dV(v,w) 

7\P) JBix(B 2 \{0}) y ' ^ ' 

7~T f X 2 (\M {1) )dV(v)x f \w n2 \ 2v \\w\\\^dV{w) 



7(V) 2{v + ani + n 2 ) 

= C v - , 

v + ani + n-i 

where in the line before the last, we have made use of the Lemma [TJ Therefore we 
see that there is a constant A > independent of v such that ||uiy|| £ 2 ^ > A. 

Now let 11 and v be distinct positive integers. We have, using the same change 
of variables (TT21): 



> z^dV(z) 




Z '\\(2)J 

X 2 (||«II (1) )<^t; !/ |HI? 2 T 1 dV(v,w) 



t~t I x 2 (\\v\\ (1) )dV(v) x I w^w^ v \\w\\ 2 ^dV{w) 



Denote the integral over B2 in this product by K. We claim that K = 0. Assuming 
this for a moment it follows that u u ) L 2 ^ = 0, and hence Hit^ — ti^H^a ^ > 

\/2A for each pair of distinct indices fi and v, and hence {u v } has no convergent 
subsequence in L§ ni (f2). The proof of the theorem is complete, provided we can 
show that K — 0. 

Let 9 be a real number which is not a rational multiple of 7r, and in the integral 
representing K, we make a change of variables w = e l6 t, where t g C™ 2 . Then, since 
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this transformation maps B2 to itself and its Jacobian is identically 1, we have 
K= / K 2 w^\\wf^dV{w) 

so that K = 0. This completes the proof. 

5. Concluding remarks 

The noncompactness of the 9-Neumann operator implies that there is a point 
in the spectrum of the Complex Laplacian \3 q acting on L\ q (fl) which is not an 
eigenvalue of finite multiplicity. It is of interest to determine the nature of this 
essential spectrum. In the case of the polydisc or product domains, the spectrum 
can be determined explicitly (see [6j [3]), and while the spectrum of a polydisc 
consists of eigenvalues only, there are infinitely many eigenvalues in the essential 
spectrum, each of infinite multiplicity (this is true, in particular, for the smallest 
nonzero eigenvalue.) It would be of interest to know whether the same is true for 
the domain H, for example in the classical situation n\ = n% = a = 1. 
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